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Accountability  *  Integrity  *  Reliability 


United  States  General  Accounting  Office 
Washington,  DC  20548 


August  16,  2002 

The  Honorable  Lane  Evans 
Ranking  Democratic  Member 
Committee  on  Veterans’  Affairs 
House  of  Representatives 

Dear  Mr.  Evans: 

For  fiscal  year  2002,  the  Department  of  Veterans  Affairs  (VA)  estimates 
that  it  will  pay  about  $25  bilhon  in  cash  disability  benefits  to  about  3.3 
million  disabled  veterans  and  their  families.  When  veterans  submit 
disability  claims,  VA’s  57  regional  offices  make  decisions  to  either  grant  or 
deny  the  requested  benefits.  Veterans  who  are  dissatisfied  with  regional 
office  decisions  may  file  appeals  with  VA’s  Board  of  Veterans’  Appeals, 
and  in  about  half  of  such  appeals,  the  Board  has  either  granted  the 
benefits  denied  by  the  regional  offices  or  remanded  (returned)  the  cases  to 
the  regional  offices  for  rework.  Additionally,  VA  reported  an  accuracy  rate 
of  less  than  70  percent  for  regional  office  disability  decisions  when  it 
tested  a  new  quality  assurance  program  in  fiscal  year  1998.  As  a  result,  we 
issued  two  prior  reports  at  your  request  on  VA’s  system  for  measuring  and 
improving  the  accuracy  of  regional  office  decisions.^ 

Recently,  questions  have  arisen  about  the  quality  of  Board  decisions. 

When  the  Board  itself  denies  benefits  requested  by  veterans,  they  may 
appeal  to  the  U.S.  Court  of  Appeals  for  Veterans  Claims.  In  over  half  of 
such  appeals,  the  court  has  either  granted  the  benefits  denied  by  the  Board 
or  remanded  the  decisions  to  the  Board  for  rework.  As  a  result,  you  now 
have  asked  that  we  examine  VA’s  efforts  to  ensure  the  quality  of  decisions 
made  by  the  Board.  Specifically,  you  asked  that  we  assess  the 
effectiveness  of  VA’s  system  for  (1)  measuring  and  improving  the  accuracy 
of  the  Board’s  decisions  and  (2)  determining  the  consistency  (extent  of 
variation)  in  decision  making  across  the  spectrum  of  regional  office  and 
Board  disability  adjudicators. 


^  U.S.  General  Accounting  Office,  Veterans’  Benefits  Claims:  Further  Improvements 
Needed  in  Claims-Processing  Accuracy,  GAO/HEHS-99-35  (Washington,  D.C.:  Mar.  1,1999), 
and  Veterans’  Benefits:  Quality  Assurance  for  Disability  Claims  Processing , 

GAO-01-930R  (Washington,  D.C.:  Aug.  23,  2001). 
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To  address  your  request,  we  reviewed  (1)  data  and  documents  of  the 
Board  of  Veterans’  Appeals;  (2)  data  and  documents  of  the  Veterans 
Benefits  Administration  (VBA),  the  VA  component  responsible  for  the 
operations  of  the  57  regional  offices;  (3)  reports  on  studies  of  VA  disability 
claims  processing  conducted  by  the  Veterans’  Claims  Adjudication 
Commission,  the  National  Academy  of  Public  Administration,  and  VA’s 
Claims  Processing  Task  Force;  and  (4)  data  reported  by  the  U.S.  Court  of 
Appeals  for  Veterans  Claims  on  its  disposition  of  veterans’  appeals.  We 
also  interviewed  officials  of  the  Board  and  VBA  about  claims  and  appeals 
processing  accuracy  and  consistency  issues.  We  conducted  our  review 
from  October  2001  through  June  2002  in  accordance  with  generally 
accepted  government  auditing  standards. 


Results  in  Brief 


In  fiscal  year  1998,  the  Board  of  Veterans’  Appeals  established  its  first 
quantitative  program  to  evaluate  and  score  its  decision-making  accuracy 
and  to  collect  data  to  identify  areas  where  the  quality  of  decision  making 
needs  improvement.  The  accuracy  measure  used  by  the  Board  understates 
its  true  accuracy  rate  because  the  Board’s  accuracy  rate  calculations 
include  certain  deficiencies,  such  as  errors  in  a  written  decision’s  format, 
which  would  not  result  in  either  a  reversal  or  a  remand  by  the  Court.  In 
fiscal  year  2001,  if  the  Board  had  excluded  format  deficiencies  from  its 
accuracy  rate  calculations,  its  accuracy  rate  would  have  been  92  percent 
versus  the  reported  rate  of  87  percent.  Even  so,  the  Board’s  quality 
assurance  program  does  not  capture  certain  data  that  potentially  could 
further  help  improve  the  quality  of  the  Board’s  decisions.  For  example,  the 
Board  does  not  record  in  its  quality  assurance  database  any  information 
identifying  the  specific  medical  issues  involved  in  cases  where  a  Board 
decision  was  judged  as  being  in  error.  Having  such  data  could  enhance  the 
Board’s  ability  to  target  training  needed  for  its  decision  makers. 

VA  does  not  assess  the  consistency  of  decision  making  across  the 
spectrum  of  regional  office  and  Board  disability  adjudicators,  even  though 
VA  acknowledges  that  in  many  cases  two  adjudicators  of  equal 
competence  could  review  the  same  evidence  but  render  different 
decisions  due  to  the  difficult  judgments  often  required  in  decision  making. 
The  result  is  that  variations  may  occur  in  the  benefits  provided  to  veterans 
who  have  similar  impairments  and  circumstances.  Even  though  available 
evidence  provides  indications  that  variations  in  decision-making  may 
occur  across  all  levels  of  VA  adjudication,  VA  does  not  conduct  systematic 
assessments  to  determine  the  degree  of  variation  that  occurs  for  specific 
impairments  and  to  provide  a  basis  for  determining  ways,  if  considered 
necessary,  to  reduce  such  variation. 
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Background 


This  report  contains  recommendations  to  the  Secretary  of  VA  concerning 
actions  to  improve  VA’s  ability  to  better  ensure  the  accuracy  and 
consistency  of  disability  decisions  made  across  the  spectrum  of  decision 
making  in  VA.  In  responding  to  a  draft  of  this  report,  VA  concurred  fully 
or  in  principle  with  our  recommendations.  However,  although  VA  agreed 
that  consistency  is  an  important  goal,  it  did  not  fully  respond  to  our 
recommendation  regarding  consistency  because  it  did  not  describe  how  it 
will  measure  consistency  and  evaluate  progress  in  reducing  any 
inconsistencies  it  may  find.  Instead,  VA  said  that  consistency  is  best 
achieved  through  comprehensive  training  and  communication  among  VA 
components  involved  in  the  adjudication  process  and  discussed  efforts 
underway  to  do  so. 


VA  has  two  basic  cash  disability  benefits  programs.  The  compensation 
program  pays  monthly  benefits  to  eligible  veterans  who  have  service- 
connected  disabilities  (injuries  or  diseases  incurred  or  aggravated  while 
on  active  military  duty).  The  payment  amount  is  based  on  the  veteran’s 
degree  of  disability,  regardless  of  employment  status  or  level  of  earnings. 
By  contrast,  the  pension  program  assists  permanently  and  totally  disabled 
wartime  veterans  under  age  65  who  have  low  incomes  and  whose 
disabilities  are  not  service-connected.^  The  payment  amount  is  determined 
on  the  basis  of  financial  need. 

VBA  and  the  Board  process  and  decide  veterans’  disability  claims  and 
appeals  on  behalf  of  the  Secretary.  The  claims  process  starts  when 
veterans  submit  claims  to  one  of  VBA’s  57  regional  offices.  (See  app.  1  for 
the  overall  flow  of  claims  and  appeals  processing.)  By  law,  regional  offices 
must  assist  veterans  in  supporting  their  claims.  For  example,  for  a 
compensation  claim,  the  regional  office  obtains  records  such  as  the 
veteran’s  existing  service  medical  records,  records  of  relevant  medical 
treatment  or  examinations  provided  at  VA  health-care  facilities,  and  other 
relevant  records  held  by  a  federal  department  or  agency.  If  necessary,  the 
regional  office  arranges  a  medical  examination  for  the  claimant  or  obtains 
a  medical  opinion  about  the  claim.  The  regional  office  adjudicator  then 
must 

analyze  the  evidence  for  each  claimed  impairment  (veterans  claim  an 
average  of  about  five  impairments  per  claim); 


^  Veterans  age  66  or  older  do  not  have  to  be  permanently  and  totally  disabled  to  become 
eligible  for  pension  benefits,  if  they  meet  the  income  and  military  service  requirements. 
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•  determine  whether  each  claimed  impairment  is  service-connected  (VA 
grants  service-connection  for  an  average  of  about  three  impairments  per 
claim); 

•  apply  VA’s  Rating  Schedule  which  provides  medical  criteria  for  rating  the 
degree  to  which  each  service-connected  impairment  is  disabling  (disability 
ratings  can  range  from  zero  to  100  percent,  in  10-percent  increments); 

•  determine  the  overall  disability  rating  that  results  from  the  combination  of 
service-connected  impairments  suffered  by  the  veteran;  and 

•  notify  the  veteran  of  the  decision. 

If  a  veteran  disagrees  with  the  regional  office’s  decision,  he  or  she  begins 
the  appeals  process  by  submitting  a  written  Notice  of  Disagreement  to  the 
regional  office.  During  fiscal  years  1999-2000,  the  regional  offices  annually 
made  an  average  of  about  616,000  decisions  involving  disability  ratings, 
and  veterans  submitted  Notices  of  Disagreement  in  about  9  percent  of 
these  decisions.  Veterans  can  disagree  with  decisions  for  reasons  other 
than  the  outright  denial  of  benefits  that  occurs,  for  example,  in  a 
compensation  case  when  a  regional  office  decides  an  impairment  claimed 
by  a  veteran  is  not  service-connected.  The  veteran  also  may  believe  the 
severity  rating  assigned  to  a  service-connected  impairment  is  too  low  and 
ask  for  an  increase  in  the  rating. 

In  response  to  a  Notice  of  Disagreement,  the  regional  office  provides  a 
further  written  explanation  of  the  decision,  and  if  the  veteran  still 
disagrees,  the  veteran  may  appeal  to  the  Board.  During  fiscal  years  1999- 
2000,  about  48  percent  of  the  veterans  who  filed  Notices  of  Disagreement 
in  decisions  involving  disability  ratings  went  on  to  file  appeals  wdth  the 
Board.  In  fiscal  year  2001,  VBA  began  nationwide  implementation  of  the 
Decision  Review  Officer  position  in  its  regional  offices.  Now,  before 
appealing  to  the  Board,  a  veteran  may  ask  for  a  review  by  a  Decision 
Review  Officer,  who  is  authorized  to  grant  the  contested  benefits  based  on 
the  same  case  record  that  the  regional  office  relied  on  to  make  the  initial 
decision.  VBA  believes  this  process  wall  result  in  fewer  appeals  being  filed 
with  the  Board. 

Located  in  Washington,  D.C.,  the  Board  is  an  administrative  body  whose 
members  are  attorneys  experienced  in  veterans’  law  and  in  reviewing 
benefits  claims.  The  Board’s  members  are  divided  into  four  decision 
teams,  with  each  team  having  up  to  15  Board  members  and  61  staff 
attorneys.  Each  team  has  primary  responsibility  for  reviewing  the  appeals 
that  originate  in  an  assigned  group  of  regional  offices.  Board  members’ 
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decisions  must  be  based  on  the  law,  regulations,  precedent  decisions  of 
the  courts,  and  precedent  opinions  of  VA’s  General  Counsel.^  During  the 
Board’s  appeals  process,  the  veteran  or  the  veteran’s  representative  may 
submit  new  evidence  and  request  a  hearing. 

During  fiscal  years  1999  and  2000,  for  all  VA  programs,  the  Board  annually 
decided  an  average  of  about  35,700  appeals,  of  which  about  32,900  (92 
percent)  were  disability  compensation  cases.  The  average  appealed 
compensation  case  contains  three  contested  issues.  As  a  result,  in  some 
cases,  the  Board  member  may  grant  the  requested  benefits  for  some  issues 
but  deny  the  requested  benefits  for  others.  During  fiscal  years  1999  and 
2000,  the  Board  in  its  initial  decisions  on  appealed  compensation  cases 
granted  at  least  one  of  the  requested  benefits  in  about  24  percent  of  the 
cases.  In  some  instances,  the  Board  member  finds  a  case  is  not  ready  for  a 
final  decision  and  returns  (or  remands)  the  case  to  the  regional  office  to 
obtain  additional  evidence  and  reconsider  the  veteran’s  claim.  During 
fiscal  years  1999  and  2000,  respectively,  the  Board  in  its  initial  decisions  on 
appealed  compensation  cases  remanded  38  percent  and  34  percent  of  the 
cases.'*  After  obtaining  additional  evidence  for  remanded  cases,  if  the 
regional  office  still  denies  the  requested  benefits,  it  resubmits  the  case  to 
the  Board  for  a  final  decision. 

If  the  Board  denies  benefits  or  grants  less  than  the  maximum  benefit 
available  under  the  law,  veterans  may  appeal  to  the  U.  S.  Court  of  Appeals 


^  According  to  VA,  precedent  opinions  of  the  General  Counsel  are  legal  opinions  that 
interpret  court  decisions,  laws,  or  regulations.  These  opinions  may  apply  such  an 
interpretation  to  a  certain  set  of  facts,  or  they  may  be  used  to  implement  an  interpretation 
of  law  consistently  across  VA.  To  the  extent  precedent  opinions  may  contain  any 
statements  of  VA  policy,  such  policy  statements  are  not  binding  on  the  Board  merely 
because  they  appear  in  a  precedent  opinion.  General  Counsel  precedent  opinions  also  are 
binding  on  regional  office  adjudicators. 

*  Effective  February  22,  2002,  a  new  VA  regulation  allows  the  Board  to  obtain  evidence, 
clarify  evidence,  cure  a  procedural  defect,  or  perform  most  any  other  action  essential  for  a 
proper  appellate  decision  without  having  to  remand  the  appeal  to  the  regional  office.  It  also 
allows  the  Board  to  consider  additional  evidence  without  having  to  refer  the  evidence  to 
the  regional  office  for  initial  consideration  and  without  having  to  obtain  the  appellant’s 
waiver.  Under  this  new  regulation,  the  Board  expects  that  the  proportion  of  appeals 
remanded  to  regional  offices  will  decline  to  about  12  percent.  For  those  cases  that 
previously  would  have  been  remanded  to  regional  offices,  the  Board  provides  information 
to  VBA  on  the  reasons  for  actions  taken  by  the  Board  itself  under  this  new  regulation,  such 
as  obtaining  new  evidence  or  clarifying  evidence. 
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for  Veterans  Claims.^  The  court  is  not  part  of  VA  and  not  connected  to  the 
Board.  During  fiscal  years  1999  and  2000,  veterans  filed  appeals  with  the 
court  in  an  estimated  10  percent  of  the  Board’s  decisions.  Unlike  the 
Board,  the  court  does  not  receive  new  evidence,  but  considers  the  Board’s 
decision,  briefs  submitted  by  the  veteran  and  VA,  oral  arguments,  if  any, 
and  the  case  record  that  VA  considered  and  that  the  Board  had  available. 
The  court  may  dismiss  an  appeal  on  procedural  grounds  such  as  lack  of 
jurisdiction,  but  in  the  cases  decided  on  merit,  the  court  may  affirm  the 
Board’s  decision  (deny  benefits),  reverse  the  decision  (grant  benefits),  or 
remand  the  decision  back  to  the  Board  for  rework.  During  fiscal  years 
1999  and  2000,  the  court  annually  decided  on  merit  an  average  of  about 
1,800  appealed  Board  decisions,  and  in  about  67  percent  of  these  cases, 
the  court  remanded  or  reversed  the  Board’s  decisions  in  part  or  in  whole.® 
Under  certain  circumstances,  a  veteran  who  disagrees  with  a  decision  of 
the  court  may  appeal  to  the  U.S.  Court  of  Appeals  for  the  Federal  Circuit 
and  then  to  the  Supreme  Court  of  the  United  States. 


The  Board 
Understates  Its 
Accuracy  and  Does 
Not  Capture  Certain 
Data  for  Improving 
Quality 


In  fiscal  year  1998,  the  Board  established  the  first  quantitative  quality 
assurance  program  to  evaluate  and  score  the  accuracy  of  its  decisions  and 
to  collect  data  to  identify  areas  where  the  quality  of  decision-making  needs 
improvement.  The  accuracy  measure  used  by  the  Board  understates  its 
true  accuracy  rate  because  the  Board’s  accuracy  rate  calculations  include 
certain  deficiencies  that  would  not  result  in  either  a  reversal  or  a  remand 
by  the  court.  Even  so,  the  Board’s  quality  assurance  program  does  not 
capture  certain  data  that  potentially  could  help  improve  the  quality  of  the 
Board’s  decisions.  Such  data  include  information  identifying  the  specific 
medical  issues  involved  in  cases  where  a  disability  decision  was  judged  as 
being  in  error.  Having  such  data  could  enhance  the  Board’s  ability  to  target 
training  for  its  decision  makers. 


®  Before  appealing  to  the  court,  a  veteran  may  ask  the  Board  itself  to  review  any  previous 
Board  decision  on  the  basis  of  “clear  and  unmistakable  error.”  During  fiscal  years  1999  and 
2000,  the  Board  revised  a  total  of  only  23  of  its  decisions  due  to  clear  and  unmistakable 
error.  VA  regulations  state  that  clear  and  unmistakable  error  is  a  specific  and  rare  kind  of 
error,  of  fact  or  law,  that  when  called  to  the  attention  of  later  reviewers  compels  the 
conclusion,  with  which  reasonable  minds  could  not  differ,  that  the  result  would  have  been 
manifestly  different  if  not  for  the  error.  Generally,  either  the  correct  facts,  as  they  were 
known  at  the  time,  were  not  before  the  Board,  or  the  statutory  and  regulatory  provisions  in 
effect  at  the  time  were  not  correctly  applied. 

®  The  court’s  Annual  Reports  did  not  disaggregate  data  on  remands  and  reversals,  but 
according  to  the  Board,  the  court  reversed  Board  decisions  in  whole  or  in  part  in  only 
about  1.6  percent  of  the  cases. 
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The  Board  Established  Its 
First  Quantitative  Quality 
Assurance  Program  in  1998 


On  the  basis  of  the  results  of  the  quality  assurance  program  it  established 
in  fiscal  year  1998,  the  Board  estimated  that  89  percent  of  its  decisions 
were  accurate  (or  “deficiency-free”).  Using  these  results  as  a  baseline,  VA 
established  performance  accuracy  goals  for  the  Board.  One  of  the  Board’s 
strategic  performance  goals  is  to  make  deficiency-free  decisions  95 
percent  of  the  time.  To  calculate  its  estimated  overall  accuracy  rate,  the 
Board  does  quality  reviews  of  selected  Board  decisions.^  We  reviewed  the 
Board’s  methods  for  selecting  random  samples  and  calculating  accuracy 
rates  and  concluded  that  the  number  of  decisions  reviewed  by  the  Board 
was  sufficient  to  meet  the  Board’s  goal  for  statistical  precision  in 
estimating  its  accuracy  rate.®  However,  we  brought  to  the  Board’s 
attention  some  issues  that  caused  the  Board  to  fall  short  of  proper  random 
sampling  and  accuracy  rate  calculation  methods,  such  as  not  ensuring  that 
decisions  made  near  the  end  of  the  fiscal  year  are  sampled  or  that  the 
results  from  quality  reviews  are  properly  weighted  in  the  accuracy  rate 
calculation  formula.  We  do  not  believe  the  overall  accuracy  rate  reported 
by  the  Board  for  fiscal  year  2001  would  have  been  materially  different  if 
these  methodological  issues  had  been  corrected  earlier;  however,  if  not 
corrected,  these  issues  potentially  could  lead  to  misleading  accuracy  rate 
calculations  in  the  future.  The  Board  agreed  in  principle  to  correct  these 
issues.  As  of  June  2002,  the  Board  had  not  yet  instituted  corrective  actions. 


The  Board’s  Actual  Level  According  to  VA’s  performance  reports,  the  Board  has  come  close  but  has 

of  Accuracy  Is  Higher  I^ot  achieved  its  annual  interim  goals  for  accuracy  (see  table  1).®  However, 

Than  Reported  calculating  its  reported  accuracy  rates,  the  Board  includes  deficiencies 

that  are  not  “substantive” — that  is,  they  would  not  be  expected  to  result  in 


’’  During  our  review,  we  brought  to  the  Board’s  attention  certain  issues  regarding  the 
compliance  of  its  quality  assurance  program  with  the  governmental  internal  control 
standard  calling  for  separation  of  key  duties  and  the  governmental  performance  audit 
standard  calling  for  organizational  independence  for  agency  employees  who  review  and 
evaluate  program  performance.  These  issues  arose  because  certain  Board  members  who 
were  directly  involved  in  deciding  veterans’  appeals  were  also  involved  in  reviewing  the 
accuracy  of  such  decisions.  Effective  May  2002,  the  Board  took  corrective  actions  to 
resolve  these  issues;  now,  all  quality  reviews  from  which  accuracy  rates  are  determined  are 
done  by  persons  not  directly  involved  in  deciding  veterans’  appeals. 

®  The  Board’s  goal  is  that  VA  can  have  96  percent  confidence  that  the  Board’s  true  accuracy 
rate  is  no  more  than  5  percentage  points  higher  or  lower  than  the  estimated  accuracy  rate. 

®  The  Government  Performance  and  Results  Act  of  1993  requires  VA,  as  well  as  other 
federal  agencies,  to  clearly  define  its  mission,  set  goals,  measure  performance,  and  submit 
to  the  Congress  annual  performance  plans  and  annual  reports  on  its  success  in  achieving 
program  performance  goals. 
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either  a  remand  by  the  court  or  a  reversal  by  the  court.  “  Consequently,  the 
reported  accuracy  rates  understate  the  Board’s  level  of  accuracy  that 
would  result  if  only  substantive  deficiencies  were  counted  in  the 
calculation. 


Table  1:  Board’s  Accuracy  Rate  Goals  and  Performance 


In  percentages 

Accuracy  rate) 

Accuracy  Measure 

FY1999 

FY  2000 

FY  2001 

FY  2002 

FY  2003 

Strategic  goal 

Accuracy  rate  goal 

90.5 

91.5 

90 

91 

92 

95 

Reported  performance  (estimated) 

84 

86 

87 

a 

"No  data  available. 

Source:  VA  budget  submissions  and  performance  plans,  fiscal  years  2000-2003. 


Under  its  quality  assurance  program,  the  Board’s  quality  reviewers  assess 
the  accuracy  of  selected  decisions  on  the  basis  of  six  critical  areas  (see 
table  2).  One  error  (or  deficiency)  in  any  of  these  six  areas  means  that  a 
decision  fails  the  quality  test.  However,  according  to  the  Board,  all  six 
areas  would  include  certain  deficiencies  that  are  not  substantive.  In 
particular,  according  to  the  Board,  most  deficiencies  in  the  “format” 
category  are  not  substantive.  In  fiscal  year  2001,  the  format  category 
accounted  for  about  38  percent  of  all  recorded  deficiencies.  At  our 
request,  the  Board  recalculated  its  accuracy  rate  for  fiscal  year  2001, 
excluding  format  deficiencies,  and  the  resulting  accuracy  rate  was  92 
percent,  as  compared  with  the  reported  accuracy  rate  of  87  percent. 
Excluding  all  other  nonsubstantive  deficiencies  presumably  would  have 
resulted  in  an  even  higher  accuracy  rate.  In  contrast  with  the  Board, 
beginning  in  fiscal  year  2002,  VBA  no  longer  includes  nonsubstantive 
deficiencies  in  its  accuracy  rate  calculations;  however,  it  continues  to 
monitor  them.  VBA  took  this  action  based  on  a  recommendation  by  the 
2001  VA  Claims  Processing  Task  Force,  which  said  that  mixing  serious 
errors  with  less  significant  deficiencies  can  obscure  what  is  of  real 
concern. 


The  court  may  set  aside  the  Board's  findings  of  fact  that  are  "clearly  erroneous."  A 
finding  is  clearly  erroneous  when,  although  there  is  evidence  to  support  it,  the  reviewing 
court  on  the  entire  evidence  is  left  with  the  definite  and  firm  conviction  that  a  mistake  has 
been  committed.  If  there  is  a  plausible  basis  in  the  record  for  the  Board's  factual 
determinations,  the  court  cannot  overturn  them,  even  if  it  might  not  have  reached  the  same 
determinations. 
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Table  2:  Board’s  Quality  Review  Criteria 

Six  critical  areas 

Criteria 

Issues 

Did  the  decision  maker  identify  and  address  all  issues  that  were  explicitly  expressed  in,  or  could  be 
inferred  from,  the  evidence? 

Evidence 

Did  the  decision  maker  account  for  all  evidence,  including  the  evidence  in  favor  of  and  the  evidence 
against  the  veteran’s  claim? 

Laws  and  regulations 

Did  the  decision  maker  cite  and  set  forth  all  applicable  laws  and  regulations? 

Reasons  and  bases 

Did  the  decision  maker  fully  and  accurately  coordinate  the  facts  (evidence)  and  laws  and  regulations  with 
each  other  and  clearly  explain  how  he  or  she  reached  the  decision? 

Due  process 

Did  the  decision  maker  address  all  technical  aspects  of  due  process  such  as  accounting  for  jurisdictional 
problems,  hearing  reguests,  and  possible  remand  reasons? 

Format 

Did  the  decision  maker  write  the  decision  so  that  it  meets  the  basic  requirements  of  format  such  as  correct 
grammar,  spelling,  decision  structure,  and  requirements  of  statute? 

Source:  Board  of  Veterans'  Appeals,  2002. 


The  Board’s  Quality  Review 
Program  Does  Not  Capture 
Certain  Data  That  Could  Help 
Improve  Quality 


The  Board’s  quality  review  program  subdivides  the  six  critical  areas 
shown  in  table  2  into  31  subcategories.  For  example,  if  a  quality  reviewer 
classifies  an  error  as  stemming  from  “reasons  and  bases,”  the  reviewer 
must  then  indicate  whether  the  error  was  due  to  misapplying  legal 
authority,  failing  to  apply  appropriate  legal  authority,  using  an  incorrect 
standard  of  proof,  or  providing  an  inadequate  explanation  for  the  decision. 
This  information  is  recorded  in  the  Board’s  quality  review  database, 
providing  the  Board  with  data  that  can  be  analyzed  to  identify  training 
needed  to  improve  quality. 


However,  the  Board  does  not  record  in  its  quality  review  database  any 
information  on  the  specific  issue  that  prompted  the  appeal  (such  as 
whether  a  disability  is  service-connected)  or  the  specific  medical 
impairment  to  which  an  error  is  related.  For  example,  a  quality  reviewer 
might  find  an  error  in  a  Board  decision  for  an  appeal  that  involved  four 
separate  medical  impairments — two  for  which  the  veteran  had  requested 
service  connection  and  two  others  for  which  he  had  requested  a  disability 
rating  increase.  On  the  basis  of  information  that  the  quality  review 
database  currently  captures,  however,  the  Board  could  not  determine 
which  of  the  four  impairments  the  error  was  related  to,  nor  could  the 
Board  determine  whether  the  error  was  related  to  a  request  for  service- 
connection  or  an  increased  disability  rating. 

This  is  not  the  case,  however,  for  Board  decisions  remanded  by  the  Court 
of  Appeals  for  Veterans  Claims.  For  these  cases,  the  Board  maintains  a 
separate  database  with  information  on  the  reasons  that  the  court  remands 
decisions  back  to  the  Board  for  rework.  For  each  issue  that  the  court 
remands  in  a  compensation  case,  the  Board  records  in  the  database  such 
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information  as:  (1)  whether  the  issue  involved  a  request  for  service- 
connection  or  an  increased  rating,  (2)  the  diagnostic  code  of  the 
impairment  involved  in  each  issue,  and  (3)  the  reason  for  the  remand. 
According  to  Board  officials,  being  able  to  analyze  the  court’s  reasons  for 
remands  by  type  of  decisional  issue  and  type  of  impairment  enhances  the 
Board’s  ability  to  reduce  remands  from  the  court  through  appropriate 
training. 

VBA  and  the  Board  recognize  that  in  some  cases,  different  adjudicators 
reviewing  the  same  evidence  can  make  differing  judgments  on  the 
meaning  of  the  evidence,  without  either  decision  necessarily  being  wrong. 
In  such  cases,  VBA  and  the  Board  instruct  quality  reviewers  not  to  record 
an  error.  A  hypothetical  case  provided  by  the  Board  furnishes  an  example. 
In  this  case,  a  veteran  files  a  claim  in  1999  asserting  he  suffered  a  back 
injury  during  military  service  but  did  not  seek  medical  treatment  at  that 
time.  One  of  the  veteran’s  statements  says  he  injured  his  back  during 
service  in  1961,  but  another  says  he  injured  his  back  in  1953.  An 
adjudicator  may  find  that  this  discrepancy  in  dates  adversely  affects  the 
claimant’s  credibility  about  whether  an  injury  actually  occurred  in  service, 
but  the  quality  reviewer  may  consider  the  discrepancy  to  be  insignificant. 
Where  such  judgments  are  involved,  the  Board’s  and  VBA’s  quality  review 
programs  recognize  that  variations  in  judgment  are  to  be  expected  and  are 
acceptable  as  long  the  degree  of  variation  is  within  reason.  (App.  11 
provides  other  examples  of  difficult  judgments  that  could  result  in 
decision-making  variations  and  explains  VA’s  “benefit-of-the-doubt”  rule.) 

The  Board  and  VBA,  however,  differ  in  their  approaches  to  collecting 
information  about  cases  where  this  type  of  variation  occurs.  In  such 
instances,  the  Board’s  quality  reviewers  note  why  they  believe  an 
alternative  decision  could  have  been  made  and  send  the  explanation  to  the 
deciding  Board  member.  However,  they  do  not  enter  any  of  this 
information  in  the  quality  review  database.  In  contrast,  VBA  recently 
instructed  its  quality  reviewers  to  enter  such  information  in  the  VBA 
quality  review  database,  even  though  no  error  is  recorded  in  the  database. 
VBA  believes  that  by  identifying  and  analyzing  cases  in  which  quality 
reviewers  believed  the  adjudicator’s  judgment  was  pushing  against  the 
boundary  of  reasonableness,  it  potentially  can  identify  opportunities  to 
improve  the  quality  of  decision  making  by  improving  training. 
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VA  Does  Not 
Systematically  Assess 
Consistency  of 
Decision  Making 


Even  though  evidence  suggests  decision  making  across  regional  office  and 
Board  adjudicators  may  not  be  consistent,  VA  does  not  systematically 
assess  decision  making  consistency  to  determine  the  degree  of  variation 
that  occurs  for  specific  impairments  and  to  provide  a  basis  for  identifying 
steps  that  could  be  taken,  if  considered  necessary,  to  reduce  such 
variation.  In  its  2003  performance  plan,  VA  acknowledged  that  veterans 
are  concerned  about  the  consistency  of  disability  claims  decisions  across 
the  57  regional  offices.  In  a  nationwide  comparison,  VBA  projected  in  its 
fiscal  year  2001  Annual  Benefits  Report  that  the  average  compensation 
payments  per  disabled  veteran  in  fiscal  year  2002  would  range  from  a  low 
of  $5,783  in  one  state  to  a  high  of  $9,444  in  another  state.  According  to  a 
VBA  official,  this  disparity  in  average  payments  per  veteran  might  be  due 
in  part  to  demographic  factors  such  as  differences  in  the  average  age  of 
veterans  in  each  state.  However,  this  disparity  in  average  payments  per 
veteran  also  raises  the  possibility  that  when  veterans  in  the  same  age 
group  submit  claims  for  similar  medical  conditions,  the  regional  office  in 
one  state  may  tend  to  give  lower  disability  ratings  than  the  regional  office 
in  another  state. 

Indeed,  in  1997,  the  National  Academy  of  Public  Administration  reviewed 
disability  claims  processing  and  said  VA  needed  to  identify  the  degree  of 
decision-making  variation  expected  for  specific  medical  issues,  set 
consistency  standards,  and  measure  the  level  of  consistency  as  part  of  the 
quality  review  process  or  through  testing  of  control  cases  in  multiple 
regional  offices.  Furthermore,  in  2001,  VA’s  Claims  Processing  Task  Force 
said  there  was  an  apparent  lack  of  uniformity  among  regional  offices  in 
interpreting  and  complying  with  directives  from  VA  headquarters  and  that 
VA’s  regulations  and  the  procedures  manual  for  regional  offices  were  in 
dire  need  of  updating.  The  task  force  concluded  that  there  was  no 
reasonable  assurance  that  claims  decisions  would  be  made  as  uniformly 
and  fairly  as  possible  to  the  benefit  of  the  veteran.  Even  though  such 
concerns  and  issues  exist,  VA  does  not  systematically  assess  the  decision¬ 
making  consistency  of  regional  office  adjudicators. 

Similarly,  VA  does  not  assess  consistency  between  decisions  made  by 
regional  offices  and  the  Board  even  though  evidence  suggests  this  issue 
may  warrant  VA’s  attention.  Because  veterans  may  submit  new  evidence 
during  the  appeals  process,  one  might  assume  that  the  Board  generally 
grants  benefits  denied  by  regional  offices  due  to  the  impact  of  such  new 
evidence.  However,  an  analysis  in  1997  of  about  50  decisions  in  which  the 
Board  had  granted  benefits  previously  denied  by  regional  offices  yielded  a 
different  viewpoint.  Staff  from  both  VBA  and  the  Board  reviewed  these 
cases  and  concluded  that  most  of  these  Board  decisions  to  grant  benefits 
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had  been  based  on  the  same  evidence  that  the  regional  offices  had 
considered  in  reaching  their  decisions  to  deny  benefits.  The  reviewers 
characterized  the  reason  for  the  Board  members’  decisions  to  grant 
benefits  as  a  difference  of  opinion  between  the  Board  members  and 
regional  office  adjudicators  in  the  weighing  of  evidence.  Furthermore, 
even  in  remanded  compensation  cases  for  which  regional  offices  have 
obtained  new  evidence  in  accordance  with  the  Board’s  remand 
instructions  and  then  again  denied  the  benefits,  the  Board  generally  has 
granted  benefits  in  about  26  percent  of  these  cases  after  they  have  been 
resubmitted  for  a  final  decision.  This  seems  to  indicate  that,  in  these 
particular  cases.  Board  members  in  some  way  differed  with  regional  office 
adjudicators  on  the  impact  of  the  new  evidence  obtained  by  the  regional 
offices  before  resubmitting  the  remanded  cases  to  the  Board. 

Available  evidence  also  provides  indications  that  the  issue  of  variations  in 
decision  making  among  the  Board  members  themselves  may  warrant  VA’s 
attention  in  studies  of  consistency.  Historically,  there  have  been  variances 
in  the  rates  at  which  the  Board’s  four  decision  teams  have  remanded 
decisions  to  regional  offices  for  rework.  No  systematic  study  has  been 
done  to  explain  the  variances  in  remand  rates.  Board  officials  said  that  it  is 
their  perception  that  the  remand  rates  vary  among  the  Board’s  decision 
teams  because  the  quality  of  claims  processing  varies  among  the  regional 
offices  for  which  each  team  is  responsible. 

Similar  concerns  about  consistency  of  claims  adjudication  in  the  Social 
Security  Administration  (SSA)  have  prompted  SSA  to  begin  taking  steps  to 
assess  consistency  issues  in  its  disability  program.  As  we  reported  in  1997, 
SSA’s  primary  effort  to  improve  consistency  has  focused  on  decision¬ 
making  variations  between  its  initial  and  appellate  levels.^^  To  gather  data 
on  variations  between  these  two  levels,  SSA  instituted  a  system  in  1993 
under  which  it  selects  random  samples  of  final  decisions  made  by 
administrative  law  judges  and  reviews  the  entire  decisional  history  of  each 
case  at  both  the  initial  and  appellate  levels.  The  reviewers  examine 
adjudicative  and  procedural  issues  to  address  broad  program  issues  such 
as  whether  a  claim  could  have  been  allowed  earlier  in  the  process.  Data 
captured  through  this  system  have  provided  a  basis  for  taking  steps  to 
clarify  decision-making  instructions  and  provide  training  designed  to 


U.S.  General  Accounting  Office,  Social  Security  Disability:  SSA  Must  Hold  Itself 
Accountable  for  Continued  Improvement  in  Decision-Making,  GAO/HEHS-97-102 
(Washington,  D.C.:  Aug.  12,  1997). 
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improve  consistency  between  the  initial  and  appellate  levels.  However,  no 
systematic  evaluations  have  been  done  to  determine  the  effectiveness  of 
these  actions.  In  its  January  2001  disabihty  management  plan,  SSA  said 
that  it  needed  to  take  further  steps  to  promote  uniform  and  consistent 
disability  decisions  across  all  geographic  and  adjudicative  levels.'^ 

Conclusions 

Opportunities  exist  to  improve  the  quality  of  the  Board’s  reporting  of 
accuracy  and  decision  making.  The  Board  includes  nonsubstantive 
deficiencies  in  its  accuracy  rate  calculation.  By  doing  so,  the  Board  may  be 
obscuring  what  is  of  real  concern.  In  addition,  the  Board’s  quality 
assurance  database  does  not  capture  data  on  specific  medical  disability 
issues  related  to  the  reasons  for  errors  found  in  Board  decisions.  Also,  in 
contrast  with  VBA,  the  Board’s  quality  assurance  program  does  not  collect 
information  on  cases  in  which  quality  reviewers  do  not  charge  errors  but 
have  differences  of  opinion  with  judgments  made  by  Board  members.  We 
believe  that  analysis  of  such  data  could  lead  to  improvements  in  quality 
through  improved  training  or  by  clarifying  regulations,  procedures,  and 
policies. 

Furthermore,  because  variations  in  decision  making  are  to  be  expected 
due  to  the  difficult  judgments  that  adjudicators  often  must  make,  one  must 
ask  the  questions:  For  a  given  medical  condition,  how  much  variation  in 
decision  making  exists  and  does  the  degree  of  variation  suggest  that  VA 
should  take  steps  to  reduce  the  level  of  variation?  VA,  however,  does  not 
assess  variation  in  decision  making.  None  of  the  quality  review  efforts  of 
either  VBA  or  the  Board  are  designed  to  systematically  assess  the  degree 
to  which  veterans  with  similar  medical  conditions  and  circumstances  may 
be  receiving  different  decisional  outcomes  or  to  help  identify  steps  that 
could  reduce  such  variation  if  necessary.  Without  ongoing  systematic 
assessments  of  consistency  across  the  continuum  of  decision  making,  VA 
cannot  adequately  assure  veterans  that  they  can  reasonably  expect  to 

Since  1994,  SSA  has  recognized  the  need  to  focus  more  attention  on  the  agency’s  overall 
quahty  assurance  program;  however,  SSA  had  made  little  progress  in  this  effort  as  of  early 
2002.  Since  then,  the  Commissioner  of  SSA  has  appointed  a  senior  manager  for  quality  who 
reports  directly  to  the  Commissioner  and  who  is  responsible  for  developing  a  proposal  to 
establish  a  quality-oriented  approach  to  all  SSA  business  processes.  See  U.S.  General 
Accounting  Office,  Social  Security  Disability:  Disappointing  Results  from  SSA’s  Efforts 
to  Improve  the  Disability  Claims  Process  Warrant  Immediate  Attention,  GAO-02-322 
(Washington,  D.C.:  Feb.  27,  2002)  and  Social  Security  Disability:  Efforts  to  Improve 

Claims  Process  Have  Fallen  Short  and  Further  Action  is  Needed,  GAO-02-826T 
(Washington,  D.C.:  June  11,  2002). 
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receive  consistent  treatment  of  their  claims  across  all  decision-making 
levels  in  VA. 


Recommendations  for 
Executive  Action 


Accordingly,  we  recommend  that  the  Secretary  of  VA  direct  the  Chairman 

of  the  Board  of  Veterans’  Appeals  to: 

•  Revise  the  quality  assurance  program  so  that,  similar  to  VBA,  the 
calculation  of  accuracy  rates  will  take  into  account  only  those 
deficiencies  that  would  be  expected  to  result  in  a  reversal  of  a  Board 
decision  by  the  U.S.  Court  of  Appeals  for  Veterans  Claims  or  result  in  a 
remand  by  the  court. 

•  Revise  the  Board’s  quality  assurance  program  to  record  information  in 
the  quality  review  database  that  would  enable  the  Board  to 
systematically  analyze  case-specific  medical  disability  issues  related  to 
specific  errors  found  in  Board  decisions  in  the  same  way  that  the  Board 
is  able  to  analyze  the  reasons  that  the  court  remands  Board  decisions. 

•  Monitor  the  experience  of  VBA’s  quality  assurance  program  in 
collecting  and  analyzing  data  on  cases  in  which  VBA’s  quality  reviewers 
do  not  record  errors  but  have  differences  of  opinion  with  regional 
office  adjudicators  in  the  judgments  made  to  reach  a  decision.  If  VBA 
finds  that  the  analysis  of  such  data  helps  identify  training  that  can 
improve  the  quality  of  decision  making,  the  Board  should  test  such  a 
process  in  its  quality  assurance  program  to  assess  whether  it  would 
enable  the  Board  to  identify  training  that  could  improve  the  quality  of 
Board  decisions. 

We  also  recommend  that  the  Secretary  direct  the  Under  Secretary  for 

Benefits  and  the  Chairman  of  the  Board  of  Veterans’  Appeals  to  jointly 


We  recognize  that  our  recommendations  will  have  to  be  implemented 
within  the  context  of  VA’s  current  major  efforts  to  reduce  a  large  and 
persistent  backlog  of  disability  claims  and  appeals  and  to  reduce  the 
average  processing  time.  Nevertheless,  we  believe  it  is  critical  that  VA  take 
the  necessary  steps  to  support  improvements  in  training  and  in 
regulations,  procedures,  or  policies  that  could  enhance  the  quality  of 
disability  decision  making  across  the  continuum  of  adjudication  and  to 
help  provide  adequate  assurance  to  veterans  that  they  will  receive 
consistent  and  fair  decisions  as  early  as  possible  in  the  process.  Indeed, 
maintaining  and  improving  quality  should  be  of  paramount  concern  while 
implementing  a  major  effort  to  reduce  backlogs  and  processing  time. 
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establish  a  system  to  regularly  assess  and  measure  the  degree  of 
consistency  across  all  levels  of  VA  adjudication  for  specific  medical 
conditions  that  require  adjudicators  to  make  difficult  judgments.  For 
example,  VA  could  develop  sets  of  hypothetical  claims  for  specific  medical 
issues,  distribute  such  hypothetical  claims  to  multiple  adjudicators  at  all 
decision-making  levels,  and  analyze  variations  in  outcomes  for  each 
medical  issue.  Such  a  system  should  provide  data  to  determine  the  degree 
of  variation  in  decision  making  and  provide  a  basis  to  identify  ways,  if 
considered  necessary,  to  reduce  such  variation  through  training  or 
clarifying  and  strengthening  regulations,  procedures,  and  policies.  Such  a 
system  should  also  assess  the  effectiveness  of  actions  taken  to  reduce 
variation.  If  departmental  consistency  reviews  reveal  any  systematic 
differences  among  VA  decision  makers  in  the  application  of  disability  law, 
regulations,  or  court  decisions,  the  Secretary  should,  to  the  extent  that 
policy  clarifications  by  VBA  cannot  resolve  such  differences,  direct  VA’s 
General  Counsel  to  resolve  these  differences  through  precedent  legal 
opinions  if  possible. 


Agency  Comments 
and  Our  Evaluation 


We  received  written  comments  on  a  draft  of  this  report  from  VA  (see  app. 
111).  In  its  comments,  VA  concurred  fully  or  in  principle  with  our 
recommendations.  With  regard  to  our  first  recommendation,  VA  said  that 
the  Board  intends  to  revise  its  quality  review  system  to  count  only 
substantive  errors  for  computational  and  benchmarking  purposes  but  will 
continue  to  track  all  errors.  On  the  basis  of  VA’s  comments,  we  also 
modified  the  report  to  accurately  reflect  the  standard  of  review  employed 
by  the  U.S.  Court  of  Appeals  for  Veterans  Claims  in  reviewing  Board 
decisions.  With  regard  to  our  second  recommendation,  VA  said  that  it 
would  use  its  Veterans  Appeals  Control  Locator  System  to  gather 
information  on  case-specific  medical  disability  issues  related  to  specific 
errors  found  in  Board  decisions.  VA  questioned  our  basis  for  concluding 
that  tracking  such  information  will  yield  useful  data  for  improving  the 
adjudication  system.  As  stated  in  the  draft  report,  we  based  our 
recommendation  on  the  fact  that  the  Board  has  already  concluded  that 
such  information  is  beneficial  for  analyzing  the  reasons  for  remands  from 
the  Court  of  Appeals  for  Veterans  Claims.  With  regard  to  our  third 
recommendation,  VA  said  representatives  of  the  Board  and  VBA  will  meet 
so  that  a  system  may  be  established  for  the  Board  to  access  and  review 
VBA’s  methodology  for  assessing,  reporting,  and  evaluating  instances  of 
“difference  of  opinion”  between  the  quality  reviewer  and  the  decision 
maker. 
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In  its  comments,  VA  concurred  in  principle  with  our  fourth 
recommendation.  VA  agreed  that  consistency  is  an  important  goal  and 
acknowledged  that  it  has  work  to  do  to  achieve  it.  However,  VA  was  silent 
on  how  it  would  measure  consistency  for  specific  medical  conditions  that 
require  adjudicators  to  make  difficult  judgments.  Instead,  VA  described 
the  kinds  of  actions  underway  that  it  believes  will  generally  reduce 
inconsistency.  While  we  support  these  efforts,  we  maintain  that  without  a 
way  to  evaluate  and  measure  consistency,  VA  will  be  unable  to  determine 
the  extent  to  which  such  efforts  actually  improve  consistency  of  decision¬ 
making  across  all  levels  of  VA  adjudication  now  and  over  time.  Neither 
will  VA  have  information  needed  to  identify  ways  to  reduce  decision¬ 
making  variations  for  specific  medical  conditions,  if  considered  necessary. 


As  agreed  with  your  office,  unless  you  publicly  announce  its  contents 
earlier,  we  plan  no  further  distribution  of  this  report  until  30  days  after  its 
issue  date.  At  that  time,  we  will  send  copies  of  this  report  to  the  Secretary 
of  the  Department  of  Veterans  Affairs,  appropriate  congressional 
committees,  and  other  interested  parties.  We  will  also  make  copies 
available  to  others  upon  request.  In  addition,  the  report  will  be  available  at 
no  charge  on  the  GAO  Web  site  at  http://www.gao.gov. 


If  you  have  questions  about  this  report,  please  call  me  on  (202)  512-7101  or 
Irene  Chu  on  (202)  512-7102.  Other  key  contributors  were  Ira  Spears,  Steve 
Morris,  Patrick  diBattista,  and  Mark  Ramage. 


Sincerely  yours. 


Cynthia  A.  Bascetta 
Director,  Education,  Workforce, 
and  Income  Security  Issues 
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Appendix  I:  Steps  in  the  Disability  Claims  and 
Appeals  Process 


Illustration  of  the 
estimated  disposition  of 
100,000  hypothetical 
compensation  claims. 


Note:  We  did  not  include  claims  and  appeals  data  for  fiscal  year  2001  in  the  basis  for  estimating  the 
disposition  of  the  100,000  cases  because  of  anomalies  related  in  whole  or  in  part  to  the 
implementation  of  the  Veterans  Claims  Assistance  Act  of  2000.  These  fiscal  year  2001  anomalies 
included  a  substantial  increase  in  the  regional  offices’  inventory  of  claims  to  be  completed  and 
substantial  increases  in  the  proportion  of  appeals  remanded  by  the  Board  and  the  Court.  See  U.S. 
General  Accounting  Office,  Veterans’  Benefits:  Despite  Recent  improvements,  Meeting  Ciaims 
Processing  Goais  Wiii  Be  Chaiienging,  GAO-02-645T  (Washington,  D.C.:  Apr.  26,  2002). 

"The  estimated  disposition  by  VA’s  regional  offices  of  the  100,000  claims  (in  boxes  1  and  2)  is  based 
on  data  for  claims  involving  disability  ratings  for  fiscal  years  1997  to  2000.  During  those  years, 
veterans  submitted  Notices  of  Disagreement  in  about  9  percent  of  the  regional  office  decisions  and 
went  on  to  file  appeals  with  the  Board  in  about  40  percent  of  the  cases  in  which  they  had  submitted 
such  notices. 

"Decisions  appealed  by  veterans  to  the  Board  reach  one  of  the  following  disposifions  before  veterans 
can  appeal  to  the  U.S.  Court  of  Appeals  for  Veterans  Claims:  the  Board  makes  an  initial  decision  to 
grant  or  deny  requested  benefits  (boxes  4  and  5);  the  regional  office  grants  requested  benefits  in 
cases  remanded  by  the  Board  for  further  development  and  reconsideration  (box  7);  the  Board  grants 
or  denies  benefits  in  remanded  cases  that  regional  offices  resubmit  to  the  Board  because  the  regional 
offices  denied  the  requested  benefits  after  developing  further  evidence  (boxes  4  and  5);  the  veteran 
withdraws  his  or  her  appeal  while  the  case  is  in  remand  status  at  the  regional  office  (box  8);  or  the 
regional  office  closes  the  case  while  in  remand  status  because  the  veteran  fails  fo  respond  to 
requests  for  information  needed  for  the  appeal  to  proceed  (box  8).  Therefore,  the  estimated 
disposition  the  3,657  appealed  cases  is  accounted  for  in  boxes  4,  5,  7,  and  8  (1,153  +  1 ,956  +  339  + 
209  =  3,657). 
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Appendix  I:  Steps  in  the  Disability  Claims  and 
Appeals  Process 


The  estimate  of  1 ,31 1  remanded  cases  (in  box  3)  is  based  on  Board  data  for  fiscai  years  1 999  and 
2000. 

"On  fhe  basis  of  Board  data  for  fiscai  years  1999  and  2000,  in  ifs  initiai  decisions  on  appeaied 
compensation  cases,  the  Board:  (1)  granted  at  ieast  one  of  the  requested  benefits  in  about  24 
percent  of  fhe  cases,  (2)  denied  aii  requested  benefits  in  about  40  percent  of  fhe  cases,  and 
(3)  remanded  about  36  percent  of  the  cases  to  regionai  offices  for  rework.  Affer  obfaining  the 
additionai  evidence  required  by  the  Board  for  remanded  cases,  the  regionai  offices  granted  requested 
benefits  in  about  22  percent  of  fhe  remanded  cases  and  denied  requested  benefits  in  64  percent  of 
fhe  cases.  After  regionai  offices  resubmitted  denied  cases  to  the  Board  for  a  finai  decision,  the  Board 
granted  at  ieast  one  of  the  requested  benefits  in  about  26  percent  of  fhe  cases,  denied  aii  benefifs  in 
about  49  percent,  and  remanded  about  25  percent  once  again  to  regionai  offices  for  further  rework. 
For  this  iiiustration,  we  assumed  that  the  Board  did  not  remand  a  case  more  than  two  times. 

°On  the  basis  of  Board  data  for  fiscai  years  1999  and  2000,  appeiiants  wifhdrew  about  13  percent  of 
remanded  compensation  cases  whiie  at  the  regionai  offices,  and  the  regionai  offices  ciosed  iess  than 
1  percent  of  the  remanded  cases  because  appeiiants  did  not  respond  to  requests  for  informafion 
needed  to  proceed  with  the  appeai. 

'The  estimate  of  307  cases  appeaied  to  the  U.S.  Court  of  Appeais  for  Veferans  Ciaims  (in  box  9),  the 
court’s  estimated  disposition  of  these  307  cases  (in  boxes  10,  11,  12),  and  the  estimated  number  of 
decisions  appeaied  fo  fhe  U.S.  Courf  of  Appeais  for  the  Federai  Circuit  (in  box  13)  are  based  on  fiscai 
years  1999  and  2000  dafa  from  fhe  court’s  annuai  reports. 

The  court’s  annuai  reports  did  not  disaggregate  data  on  reversais  and  remands.  According  to  the 
Board,  the  court  reversed  Board  decisions  in  whoie  or  in  part  in  oniy  about  1 .6  percent  of  fhe  cases 
during  fiscai  years  1999  and  2000. 

The  Board  did  not  have  any  data  that  wouid  provide  a  basis  for  estimating  the  number  of  granfs  and 
deniais  made  by  fhe  Board  or  regionai  offices  in  cases  remanded  by  fhe  court  (in  box  14). 

Source:  Prepared  by  GAO  using  data  from  VBA,  Board  of  Veterans’  Appeals,  and  the  U.S.  Court  of 
Appeals  for  Veterans  Claims. 


Page  18 


GAO-02-806  Veterans’  Disability  Benefits 


Appendix  II:  Board  of  Veterans’  Appeals 
Illustrations  of  Difficult  Judgments  Resulting 
in  Decision-Making  Variations 


Adjudicator’s  task 

Examples  of  difficult  judgments 

Assessing  credibility  of  sources  of 
evidence 

To  be  granted  benefits  for  post-traumatic  stress  disorder,  a  veteran’s  claim  must  have 
credible  evidence  that  a  stressor  occurred  during  military  service.  Assume  the  record 
shows  a  claimant  served  in  Vietnam  as  a  supply  specialist,  and  he  identified  mortar 
attacks  as  a  stressor.  Reports  prepared  by  his  military  unit  in  Vietnam  indicate  a  single 
enemy  mortar  attack  occurred  where  the  claimant  was  stationed.  The  claimant’s 
testimony  was  vague  about  the  number  and  the  time  of  the  attacks.  One  adjudicator  may 
rely  on  the  unit’s  reports  and  conclude  the  claimant  engaged  in  combat  and  is  entitled  to 
have  his  lay  statements  accepted  without  further  corroboration  as  satisfactory  evidence 
of  the  in-service  stressor.  Another  adjudicator  may  conclude  that  the  claimant  is  not 
credible  as  to  exposure  to  enemy  fire  and  require  other  credible  supporting  evidence  that 
the  in-service  stressor  actually  occurred. 

Evaluating  and  assigning  weight  to 
evidence 

Assume  an  appeal  for  either  service  connection  or  a  higher  disability  rating  has  two 
conflicting  medical  opinions,  one  provided  by  a  medical  specialist  who  reviewed  the  claim 
file  but  did  not  actually  examine  the  veteran  and  a  second  opinion  provided  by  a  medical 
generalist  who  reviewed  the  file  and  examined  the  veteran.  One  adjudicator  could  assign 
more  weight  to  the  specialist’s  opinion,  while  another  could  find  the  generalist’s  opinion 
to  be  more  persuasive.  Thus,  depending  on  which  medical  opinion  is  given  more  weight, 
one  adjudicator  could  grant  the  claim  and  the  other  deny  it.  Yet,  a  third  adjudicator  could 
find  both  opinions  to  be  equally  probative  and  conclude  that  VA’s  “benefit-of-the-doubt” 
rule  requires  that  he  decide  in  favor  of  the  veteran’s  request  for  either  service-connection 
or  a  higher  disability  rating.  Under  the  benefit-of-the-doubt  rule,  if  an  adjudicator 
concludes  that  there  is  an  approximate  balance  between  the  evidence  for  and  the 
evidence  against  a  veteran’s  claim,  the  adjudicator  must  decide  in  favor  of  the  veteran. 

Applying  subjective  standards  in  VA’s 
Rating  Schedule 

The  Rating  Schedule  does  not  provide  objective  criteria  for  rating  the  degree  to  which 
certain  spinal  impairments  limit  a  claimant’s  motion.  The  adjudicator  must  assess  the 
evidence  and  draw  a  conclusion  as  to  whether  the  limitation  of  motion  falls  into  one  of 
three  severity  categories:  “slight,  moderate,  or  severe.”  Similarly,  in  assessing  the 
severity  of  incomplete  paralysis,  the  adjudicator  must  draw  a  conclusion  as  to  whether 
the  veteran’s  incomplete  paralysis  falls  into  one  of  three  severity  categories:  “mild, 
moderate,  or  severe.”  In  each  case,  each  severity  category  in  itself  encompasses  a 
range  of  severity,  and  the  judgment  as  to  whether  a  claimant’s  condition  is  severe 
enough  to  cross  over  from  one  severity  range  into  the  next  could  vary  in  the  minds  of 
different  adjudicators. 

The  Rating  Schedule  provides  a  formula  for  rating  the  severity  of  a  veteran’s 
occupational  and  social  impairment  due  to  a  variety  of  mental  disorders.  However,  the 
formula  actually  is  a  nonquantitative,  behaviorally  oriented  framework  for  guiding 
adjudicators  in  making  judgments  and  drawing  conclusions  as  to  which  of  the  following 
characterizations  best  describes  the  degree  to  which  a  claimant  is  occupationally  and 
socially  impaired:  (1)  totally  impaired;  (2)  deficient  in  most  areas  such  as  work,  school, 
family  relations,  judgment,  thinking,  or  mood;  (3)  reduced  reliability  and  productivity; 

(4)  occasional  decrease  in  work  efficiency  and  intermittent  periods  of  inabiiity  to  perform 
occupational  tasks;  (5)  mild  or  transient  symptoms  that  decrease  work  efficiency  and 
ability  to  perform  occupational  tasks  only  during  periods  of  significant  stress  or  symptoms 
can  be  controlled  by  continuous  medication,  and  (6)  not  severe  enough  to  interfere  with 
occupational  or  social  functioning  or  to  require  continuous  medication. 

Source:  Board  of  Veterans'  Appeals. 
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Appendix  III:  Comments  from  the 
Department  of  Veterans  Affairs 


THE  SECRETARY  OF  VETERANS  AFFAIRS 
WASHINGTON 

July  31 , 2002 


Ms.  Cynthia  A.  Bascetta 

Director,  Veterans  Health  and  Benefits  Issues 

Education,  Workforce  and  Income  Security  Team 

U.  S,  General  Accounting  Office 

441  G  Street,  NW 

Washington,  DC  20548 

Dear  Ms.  Bascetta: 

The  Department  of  Veterans  Affairs  (VA)  has  reviewed  your  draft  report, 
VETERANS’  BENEFITS:  Quality  Assurance  for  Disability  Ciaims  and 
Appeais  Processing  Can  Be  Further  improved  (GAO-02-806).  VA 
appreciates  that  the  General  Accounting  Office  (GAO)  supports  VA’s  efforts  at 
maintaining  and  improving  quality  while  implementing  a  major  effort  to  reduce 
backlogs  and  processing  time  of  benefits  claims. 

VA  concurs  with  GAO's  recommendations  that  the  Board  of  Veterans’ 
Appeals  (BVA)  (a)  revise  its  calculation  of  accuracy,  (b)  revise  its  recording  of 
data,  and  (c)  monitor  the  Veterans  Benefits  Administration’s  (VBA)  experience 
with  recording  data  as  to  difference  of  opinion  between  the  decision  maker  and 
the  quality  reviewer.  VA  also  concurs  in  principle  with  GAO's  recommendation  to 
establish  a  system  to  regularly  assess  and  measure  the  degree  of  consistency  of 
adjudication  decisions.  The  Department  agrees  consistency  is  an  important  goal. 
However.  VA  believes  this  is  best  achieved  through  comprehensive  training  and 
communication  among  VA's  components  involved  in  the  adjudication  process. 

The  enclosure  provides  detailed  comments  to  each  of  GAO’s 
recommendations.  Thank  you  for  the  opportunity  to  comment  on  your  draft 
report. 


Sincerely  yours, 


Enclosures 
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Appendix  III:  Comments  from  the  Department 
of  Veterans  Affairs 


Enclosure 

THE  DEPARTMENT  OF  VETERANS  AFFAIRS  COMMENTS  TO 
GAO  DRAFT  REPORT,  VETERANS’ BENEFITS;  Quality  Assurance  for 
Disability  Claims  and  Appeals  Processing  Can  Be  Further  Improved 

(GAO-02-806) 

GAO  recommends  that  I  direct  the  Chairman  of  the  Board  of  Veterans’ 
Appeals  to: 

•  Revise  the  quality  assurance  program  so  that,  similar  to  VBA,  the 
calculation  of  accuracy  rates  will  take  into  account  only  those 
deficiencies  that  would  be  expected  to  result  in  a  reversal  of  a 
Board  decision  by  the  U.S.  Court  of  Appeals  for  Veterans  Claims 
due  to  clear  and  unmistakable  error  or  result  in  a  remand  by  the 
Court. 

Concur  in  Principle  -  The  Board  intends  to  revise  its  quality  review  system  to 
count  only  substantive  errors  for  computational  and  benchmarking  purposes,  but 
continue  to  track  all  errors.  It  is  considering  excluding  the  category  of  “formaf' 
errors  from  the  calculation  of  its  accuracy  rate,  at  least  as  an  interim  measure. 

On  page  8  of  its  draft  report,  GAO  cites  this  as  raising  BVA’s  accuracy  rate  for 
FY  2001  from  87  percent  to  92  percent  and  providing  a  more  useful  assessment 
of  BVA's  decision  accuracy.  Since  the  Board  has  accumulated  the  underlying 
data  for  past  years,  it  can  recalculate  past  years’  performance  on  this  basis,  and 
thereby  continue  to  provide  a  solid  basis  for  benchmarking  BVA’s  quality. 

However,  the  Court  does  not  reverse  the  Board  on  the  basis  of  clear  and 
unmistakable  error.  As  GAO’s  characterization  of  the  standard  of  review  by  the 
Court  is  erroneous,  VA  cannot  concur  with  the  recommendation  as  stated. 


•  Revise  the  Board’s  quality  assurance  program  to  record 

information  in  the  quality  review  database  that  would  enable  the 
Board  to  systematically  analyze  case-specific  medical  disabilKy 
Issues  related  to  specific  errors  found  in  Board  decisions  in  the 
same  way  that  the  Board  is  able  to  analyze  the  reasons  that  the 
Court  remands  Board  decisions. 

Concur  -  Implementing  this  recommendation  would  fall  within  the  capabilities  of 
VA’s  Veterans  Appeals  Control  Locator  System  (VACOLS).  In  order  to 
accomplish  this,  the  Board  will  consult  with  IT  staff  to  determine  the  technical 
feasibility  of  doing  so  and  then  develop  a  specific  implementation  plan. 
Nevertheless,  VA  questions  the  report’s  basis  for  concluding  that  such  tracking 
will  yield  useful  data  for  improvement  of  the  adjudication  system,  as  GAO 
provides  no  empirical  data  to  support  its  assertion  that  this  would  be  a  valuable 
exercise. 
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Appendix  III:  Comments  from  the  Department 
of  Veterans  Affairs 


Enclosure 

THE  DEPARTMENT  OF  VETERANS  AFFAIRS  COMMENTS  TO 
GAO  DFIAFT  REPORT.  VETERANS’ BENEFITS:  Quality  Assurance  for 
Disability  Ciaims  and  Appeais  Processing  Can  Be  Further  improved 
(GAO-02-806) 

(Continued) 


»  Monitor  the  experience  of  VBA’s  quaiity  assurance  program  in 
collecting  and  analyzing  data  on  cases  in  which  VBA’s  quality 
reviewers  do  not  record  errors  but  have  differences  of  opinion 
with  regional  office  adjudicators  in  the  judgments  made  to  reach 
a  decision.  If  VBA  finds  that  the  analysis  of  such  data  helps 
identify  training  that  can  improve  the  quality  of  decision-making, 
the  Board  should  test  such  a  process  in  its  quality  assurance 
program  to  assess  whether  it  would  enable  the  Board  to  identify 
training  that  could  improve  the  quality  of  Board  decisions. 

Concur  -  Representatives  of  the  Board  and  VBA  will  meet  so  that  a  system  may 
be  established  for  the  Board  to  access  and  review  VBA's  methodology  for 
assessing,  reporting,  and  evaluating  instances  of  “difference  of  opinion”  between 
the  quality  reviewer  and  the  decision  maker. 


GAO  also  recommends  that  I  direct  the  Under  Secretary  for  Benefits  and 
the  Chairman  of  the  Board  of  Veterans’  Appeals  to  jointly  establish  a 
system  to  regularly  assess  and  measure  the  degree  of  consistency 
across  all  levels  of  VA  adjudication  for  specific  medical  conditions  that 
require  adjudicators  to  make  difficult  judgments.  For  example,  VA 
could  develop  sets  of  hypothetical  claims  for  specific  medical  issues, 
distribute  such  hypothetical  claims  to  multiple  adjudicators  at  all 
decision-making  levels,  and  analyze  variations  in  outcomes  for  each 
medical  issue.  Such  a  system  should  provide  data  to  determine  the 
degree  of  variation  in  decision-making  and  provide  a  basis  to  identify 
ways,  if  considered  necessary,  to  reduce  such  variation  through 
training  or  clarifying  and  strengthening  regulations,  procedures,  and 
policies.  Such  a  system  should  also  assess  the  effectiveness  of  actions 
taken  to  reduce  variation.  If  departmental  consistency  reviews  reveal 
any  systematic  differences  among  VA  decision-makers  in  the 
application  of  disability  law,  regulations,  or  court  decisions,  the 
Secretary  should,  to  the  extent  that  policy  clarifications  by  VBA  cannot 
resolve  such  differences,  direct  VA’s  General  Counsel  to  resolve  these 
differences  through  precedent  legal  opinions  if  possible. 
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Appendix  III:  Comments  from  the  Department 
of  Veterans  Affairs 


Enclosure 

THE  DEPARTMENT  OF  VETERANS  AFFAIRS  COMMENTS  TO 
GAO  DRAFT  REPORT.  VETERANS’  BENEHTS:  Quality  Assurance  for 
Disability  Claims  and  Appeals  Processing  Can  Be  Further  Improved 
(GAO-02-806) 

(Continued) 


Concur  in  Principle  -  VA  agrees  that  consistency  is  an  important  goal,  and  the 
Department  has  work  to  do  to  achieve  it.  However,  VA  believes  improved 
consistency  is  best  achieved  by  comprehensive  training  and  communication 
among  the  VA  components  involved  in  the  adjudication  process.  There  are 
significant  individual  and  joint  training  efforts  underway  in  BVA,  VBA,  the  Office 
of  General  Counsel,  and  the  Veterans  Health  Administration  to  improve  the 
quality  and  consistency  of  the  adjudication  process.  For  example,  VBA  is  using 
the  national  Systematic  Technical  Accuracy  Review  (STAR)  to  promote  quality 
and  consistency.  In  the  near  future,  VBA  will  conduct  Systematic  Individual 
Performance  Assessments  (SIPA)  for  all  its  employees.  In  addition,  VBA’s 
national  training  program,  including  its  computer-based  Training  and 
Performance  Support  Systems  (TPSS)  program,  provides  a  uniform  approach 
toward  training  new  employees.  Finally,  the  creation  of  specialized  teams  in 
the  Claims  Processing  Improvement  (CPI)  model,  coupled  with  changes  to  the 
Rating  Schedule,  will  undoubtedly  provide  more  consistent  decisions  through 
repetition  in  dealing  with  specific  medical  conditions. 
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GAO’S  Mission 

The  General  Accounting  Office,  the  investigative  arm  of  Congress,  exists  to 
support  Congress  in  meeting  its  constitutional  responsibilities  and  to  help 
improve  the  performance  and  accoimtability  of  the  federal  government  for  the 
American  people.  GAO  examines  the  use  of  public  funds;  evaluates  federal 
programs  and  policies;  and  provides  analyses,  recommendations,  and  other 
assistance  to  help  Congress  make  informed  oversight,  policy,  and  fimding 
decisions.  GAO’s  commitment  to  good  government  is  reflected  in  its  core  values 
of  accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost  is 
through  the  Internet.  GAO’s  Web  site  (www.gao.gov)  contains  abstracts  and  full- 
text  files  of  current  reports  and  testimony  and  an  expanding  archive  of  older 
products.  The  Web  site  features  a  search  engine  to  help  you  locate  documents 
using  key  words  and  phrases.  You  can  print  these  documents  in  their  entirety, 
including  charts  and  other  graphics. 

Each  day,  GAO  issues  a  list  of  newly  released  reports,  testimony,  and 
correspondence.  GAO  posts  this  list,  known  as  “Today’s  Reports,”  on  its  Web  site 
daily.  The  list  contains  links  to  the  full-text  document  files.  To  have  GAO  e-mail 
this  list  to  you  every  afternoon,  go  to  www.gao.gov  and  select  “Subscribe  to  daily 
E-mail  alert  for  newly  released  products”  imder  the  GAO  Reports  heading. 

Order  by  Mail  or  Phone 

The  first  copy  of  each  printed  report  is  free.  Additional  copies  are  $2  each.  A 
check  or  money  order  should  be  made  out  to  the  Superintendent  of  Documents. 
GAO  also  accepts  VISA  and  Mastercard.  Orders  for  100  or  more  copies  mailed  to  a 
single  address  are  discounted  25  percent.  Orders  should  be  sent  to: 

U.S.  General  Accounting  Office 

441  G  Street  NW,  Room  LM 

Washington,  D.C.  20548 

To  order  by  Phone:  Voice:  (202)  512-6000 

TDD:  (202)  512-2537 

Fax:  (202)  512-6061 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 
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